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With the increase use of the Internet, distance education will continue to provide a vehicle for instruction 
and/or training. Traditional methods for measuring quality instruction and training may need to be 
reviewed for their applicability and use with current technological methods of instructional delivery. This 
paper reviews literature on the current evaluation of quality in distance education and the implications for 
further research. 
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Distance Education has been a form of instruction in the United States for over 110 years (Moore, 2003). Some of 
the first courses were delivered through correspondence to provide “collegiate instruction and to award diplomas 
and degrees” (Moore, 2003, p. 11) regardless of the separation between student and teacher. Since then distance 
education evolved and has utilized different delivery strategies and technologies. Technological advances provided 
modes of delivery such as public television, compressed digital media, and on-line courses. Entire programs of 
study are available at a distance. Academic institutions of higher learning use specialized software or course 
management systems to provide students courses and/or degree programs delivered on-line. 

During 2000-2001 “56 percent of all postsecondary institutions (within the United States) offered distance 
education courses” (The Condition of Education, 2004, p. 85). According to UCLA’s Internet report (2001) it found 
that 72.3 percent of Americans go online and that they are most satisfied with the Internet’s ability to help them 
communicate with other people” (Meyer, 2002, p. 3) Additionally “87 percent of campuses provide college 
applications on-line, 55 percent provide course registration over the web and 56 percent offer fully on-line courses” 
(Meyer, 2002, p. 4) With increased need for new career skills and improvement in delivery technology, distance 
education students will demand evidence of quality and authenticity in distance courses” (Cavanaugh, 2002, p. 189). 
This paper will address the need to operationalize “quality” to help prospective learners, program planners, and 
designers ensure that distance programs, courses, or training adhere to appropriate standards. 

Overview of Distance Education 

The foundational purpose for distance education was to allow remote students to study at home. “From 1883 to 
1891, academic degrees were authorized by the state of New York through the Chautauqua College of Liberal Arts 
to students who completed the required summer institutes and correspondence courses” (Simonson, Smaldino, 
Albright, Zavcek, 2000, p. 32) These pioneer efforts evolved to delivery methods and media that are the current 
status of technology-based instruction and training. A survey by the U.S. Department of Education's National 
Center for Educational Statistics (NCES) found that from 1994-95 to 1997-98 the number of distance education 
degree programs increased by 72%. Moreover, an additional 20 percent of the institutions surveyed plan to establish 
distance education programs within the next three years (The Institute for Higher Education Policy, 2000). This 
surge of distance education prompted the need for building a case for quality in the courses, degree and training 
programs offered by private and public institutions. 

Formal educational institutions are not the only organizations that envision the potential for distance education. 
It is predicted that in the next 5 years, 95% of all workers will use some form of information technology in their jobs 
(Simonson, Smaldino, Albright, Zavcek, 2000). The Commission on Technology and Adult Learning foresees a 
future in which e-learning (distance education) will become a continuous process of inquiry and improvement that 
keeps pace with the speed of change in business and society. With e-learning, the learner has convenient, just-in- 
time access to needed knowledge and information, with small content objects assembled and delivered according to 
the learners specific needs (American Society for Training and Development, 2001). Organizations are looking to 
e-learning or distance education to provide training and development for their employees to get up to speed on new 
products and processes. Improvements in the quality of education and training are an equally important economic 
benefit of e-learning, which offers potentially universal access to best-in-class learning content as well as a wide 
variety of content available anywhere in the world (American Society for Training and Development, 2001). 
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While the Commission on Technology and Adult Learning realized the potential of distance education, it also 
realizes that assuring quality training is essential in the development of their employees. Corporations save between 
50-70% when replacing instructor-led training with electronic content delivery. Opting for e-learning also means 
that courses can be pared into shorter days so that the business does not loose an employee for entire days at a time 
(American Society for Training and Development, 2001). The Commission realizes that this will only be possible 
if both pubic and private sectors engage in quality distance education training programs. But what is meant by 
quality and how can it be evaluated? 

Theoretical Framework 

The earliest distance education comparison study that demonstrated that there was “no significant difference” in 
student achievement was that of L.T. Russell in 1999. “Russell reviewed 355 studies on distance education 
produced from 1928 to 1998 that included comparison studies of instruction using videotape, interactive video, 
satellite, telecourses, television with on-campus, and in-person courses” (Meyer, 2002, p. 13). His work 
demonstrated that it was not the technology that enhanced learning; it was the instructional method used through the 
media. 

There have been additional comparative studies based on Russell’s foundational work as well as several 
interpretations of his work. During the same year his research was criticized for its lack of “quality due to the lack 
of control groups, randomization of treatment groups, matching of student populations, statistical sophistication, and 
consistency in treatments” (Meyer, 2002, p. 16). Additionally he focused on “courses and not on programs, did not 
report on differences in students especially their learning styles, the interaction of multiple technologies, and not 
basing research on theoretical frameworks” (Meyer, 2002, p. 16). 

During the same year, at a National Convention of the Association for Educational Communications and 
Technology (AECT) in Houston, Texas, Simonson presented the emergence of a new theory in distance education. 
The equivalence theory was stated as “distance education’s appropriate application that should provide equivalent 
learning experiences for all students distant and local - in order for there to be expectations of equivalent outcomes 
of the educational experience” (Simonson, 1999, p. 209). The essence of this theory states that learners, both distant 
and local, do have different learning environments. The design of the instruction should provide learning 
experiences that are of the same value, although the experiences may differ slightly. This theory provides the base 
of “core values such as local control and personalized instruction that are held almost sacred in classical American 
education” (Simonson, 1999, p. 209). Simonson goes on to state that if the acceptance of distance education is to 
occur, it must be “high quality easily obtained, and familiar to those in need” (Simonson, 1999, p. 209). Although 
his presentation indicated that there is more than one foundational theory for distance education, he did indicate that 
the equivalency theory provided a framework for instruction that gives distant or local learners the equivalent 
learning experiences. The equivalency theory provides the understanding that while the environment is different; the 
learning experiences should be of equal value, hence equal in the quality of instruction through distance education. 

Problem Statement 

Currently there is no universal agreement on the definition of quality in distance education. While there is no 
standard definition there is a concern that “the focus on the quality of distance education has become an emotional 
and political issue, with opponents arguing that the new use of technology is, by definition, of poor quality and 
proponents arguing that on-line work can produce valuable student learning” (Meyer, 2002, p. 7). 

Purpose of the Study 

Distance education can be defined in many ways — it can mean geographical distance, time distance, and even 
intellectual distance (Simonson, Smaldino, Albright, Zavcek, 2000). For the purpose of this paper, distance 
education will be defined as “institutional based, formal education where the learning group is separated 
geographically, and where interactive telecommunication systems are used to connect learners, resources and 
instructors” (Simonson, Smaldino, Albright, Zavcek, 2000, p. 8). Quality also has multiple meanings. “The 
meaning of quality differs depending upon the circumstances and perceptions; quality is also time based or 
situational” (Oakes & Wescott, 2001: p. 350). This study will reveal the meaning of quality in distance education 
based upon a review of the literature. The following research questions guided this study: 

1 . How is quality evaluated in distance education? 
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2. What are the current definitions of quality in distance education programs? 

Methodology, Methods and Scope 

The method used to review the literature on distance education, evaluation and quality was to search scholarly 
electronic databases, journal publications, books and Internet sites for relevant information from 1986 to 2003. The 
information gathered was analyzed for its usefulness in answering the research questions concerning this study. The 
electronic databases accessed included: ABI Inform, ERIC, and ERIC Full Text Documents. Electronic journals 
accessed were the American Journal of Distance Education, American Journal of Evaluation, and the Quarterly 
Review of Distance Education. Internet sites reviewed for information concerning distance education were the 
United States Department of Education and the Texas Higher Education Coordinating Board. The combination of 
searches supplied a total of 387 articles and 87 articles were gleaned for their relevance to distance education and 
quality for review. After analyzing the sources for relevant information, the results provided a total of 23 articles of 
relevant information concerning distance education, quality, and evaluation. 

The limitations of this study are that sources for quality in distance education were primarily in the United 
States with one source from international literature assessing for quality in distance education. The literature 
reviewed was based on distance education, quality and evaluation keyword searches, resulting in articles primarily 
concerned with quality in higher education. Additional keywords for distance education that could be used are e- 
learning, web-based learning and on-line learning to expand on this preliminary review. The incorporation of 
international sources from countries such as the United Kingdom and Australia could provide further insight. 

Findings for Quality in Distance Education 

The preliminary review of the literature revealed that evaluation of quality is being conducted at the institutional, 
course, program, instructor and student levels. The institutional level of evaluation is often aligned with state and 
federal accreditation. 

Institutions offering courses or programs of study through distance education follow the traditional on-site 
classroom evaluation techniques; however, they are required to demonstrate additional efforts for quality by 
adhering to state and federal policies and regulations. These policies and regulations are provided by what has 
traditionally been called the “Triad, the parties to quality, which includes the states, the federal government and the 
accrediting community” (Southern Regional Education Board, 2002, p. 4). Educational institutions for higher 
learning are assessing quality based primarily on state policies developed in the early 1990’s by the Western 
Cooperative for Educational Telecommunications in a publication entitled Principles of Good Practice for 
Electronically Offered Academic Degree and Certificate Programs which has been widely circulated and adopted 
by states, regional accrediting associations and others (Southern Regional Education Board, 2002). An example of 
this adoption is the Texas Higher Education Coordinating Board Guide for Incorporating the Principles of Good 
Practice into Electronically-Based Courses. It assesses quality in the areas of curriculum and instruction, 
institutional context and commitment, evaluation, and faculty and institutional commitment. 

The most common forms of quality evaluation are formative and summative techniques, which are typically 
conducted towards the end of the development and delivery process (Sims, Dobbs & Hand, 2002). At the course 
and program level, post-evaluations gauge the student’s evaluation of the course on-line, student achievement, and 
course effectiveness (based on student’s perception). Faculty members and co-workers are to use the results to 
make necessary revisions or changes at regular internals. This post-evaluation aligns itself with traditional on-site 
classroom end of course evaluations. Other methods for determining quality within the course or degree program are 
student grades, student retention during the semester, and the number of students withdrawing from the course 
during the semester (attrition rate). 

One source promoted the idea of proactive evaluation for enhancing quality in on-line learning (Sims, Dobbs & 
Hand, 2002). The authors contend that conventionally, evaluation has been positioned at the end of the instructional 
development cycle, to assess whether or not the creative effort achieved the original product goals and whether or 
not the desired learning outcomes were realized (Sims, Dobbs & Hand, 2002). This proactive evaluation advocates 
planning and design to ensure that all areas of learning are addressed and to provide for a more meaningful and 
directed post evaluation. “By focusing on the planning and design phase of the development cycle, proactive 
evaluation addresses the critical issues associated with the creation of learning resources and environments for 
delivery in an on-line context to ensure they will have a greater chance of achieving educational outcomes” (Sims, 
Dobbs, & Hand. 2002, p. 147). Table 1 below highlights the articles that were used to determine where quality was 
being evaluated and their dependent variables. 
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Table 1. Areas Evaluated for Quality and Dependent Variables 



Year 


Author 


Areas Evaluated for Quality 


Dependent Variable(s) 


1987 


Kember & Harper 


Student 


Student Learning and Learning Styles 


1990 


Kember 


Student 


Student Support 


1990 


Thompson 


Student and Instructor 


Interaction 


1998 


Zuniga 


Accreditation 


Academic Content 
Academic Resources 
Student Retention 
Technological Opportunities and 
Challenges 


1999 


Wentling & 

Johnson 


Instruction 


Student Demand 
Student Retention 
Student Satisfaction 
Faculty Satisfaction 
Student Achievement 
Financial Efficiency 


2000 


Institute for Higher 
Learning 


Accreditation 


Institutional Support 
Teaching/Learning 
Student Support 


2001 


Twigg 


Accreditation 


Institutional Support 
Course Development 
Teaching/Learning 
Course Structure 
Student Support 


2001 


ASTD 


Program 


Student/Employee Experience 
Student Centered 


2002 


Schoening 


Student 


Student Learning 


2002 


Roblyer & 

Wiencke 


Course 


Interaction 


2002 


Thurmond. 
Wamback, Conners 
& Frey 


Coins e 


Student Satisfaction 
Student Learning 

Student Pre and Post Evaluation of 
Knowledge and Skills 


2002 


Sims 


Program 


Pre-Evaluation of Program Planning and 
Design 


2002 


Howland & Moore 


Student 

Instructor 


Student Experiences 
Instructor Experiences 


2002 


CHEA 


Institutional 


Student Success 
Policies 
Standards 
Procedures 


2002 


Cavanaugh 


Program 


Program Design 

Resources 

Practices. 


2002 


Allen 


Student 


Student Satisfaction 
Interaction 


2002 


Bisciglia 


Student 


Student Attitudes 


2002 


Meyer 


Courses 


Interaction 


2002 


SREB 


Accreditation 


Federal 

Institutional Context and Commitment 
Curriculum and Instruction 


2003 


Benson 


Accreditation 


Institutional Context and Commitment 
Curriculum and Instruction 
Student Support 
Evaluation and Assessment 
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Table 1. Areas Evaluated for Quality and Dependent Variables Continued 



Year 


Author 


Areas Evaluated for Quality 


Dependent Variable(s) 


2003 


Williams 


Programs 


Roles and Competencies needed by DE 
Professionals 


2003 


Ciaverelli 


Teachers 


Instructional 






Institutional 


Student Success 



While the information provided in the articles addressed specific areas for evaluation, each area had their own 
specific dependent variable or variables. During the course of this literature review it was determined that there was 
no common definition for quality in distance education. Quality is evaluated at different levels. The definition 
changes depending upon the level at which quality is being measured and/or addressed. The following themes for 
meanings of quality in distance education include: 

• Quality is meeting accreditation standards (Benson, 2003). 

• Quality is utilizing tools such as interaction, self-examination, and student evaluations (Hansen, 2003). 

• Quality is effective and appropriate instruction (Cavanaugh, 2002). 

• Quality is emphasis on the individual learner (Fenwick, 1992). 

Many recommendations and statements concerning quality in distance education were also noted and are provided 
below: 

• The Southern Regional Education Board through the Electronic Campus and Data Exchange initiatives 
should work to establish common definitions (SREB, 2002). 

• Since we have not achieved a definitive answer on quality for more traditional classroom situations, 
perhaps it is unwise to expect such clarity for on-line learning. However, more understanding is always 
better, so the search for clarity will continue (Meyer, 2002). 

• Assuring the future quality of adult learning must be a priority as government and business work together 
to build America’s e-learning future (ASTD, 2001). 

The following theories emerged from the literature during the review. Although these are not the only theories 
associated with distance education, these were found during the preliminary review of the literature. 

Distance Education 

Although the history of distance education extends over 100 years, it was not until 1986 that Borg Holemberg 
began to voice his concerns of the lack of theories explaining, and identifying distance education. His concern was 
due to his belief that distance education was not a “deviation of conventional education, he claimed that it was a 
distinct field of education, parallel and a complement of conventional education” (Schlosser & Anderson, 1994, p. 
6). 

Independent Study 

Charles Wedemeyer proposed the concept of independent study based on the essence of the separation between 
student and teacher, teaching and learning, and student’s pace in learning (Schlosser & Anderson, 1994). Michael 
Moore also noted the gap between student and instructor in distance education. This distance shifted the 
responsibility of learning to the student. Although students are still able to rely on their instructors for guidance, due 
to the distance between them the student takes on a more active role in learning than in a traditional classroom. 
Interaction and Communication 

The theories informing interaction and communication in distance education can be linked to several 
disciplines. The “most useful model of interactive qualities for distance learning provide two insights: (1) 

characteristics that define interaction in distance learning and (2) factors that influence it in distance learning 
settings” (Roblyer & Wienke, p. 79). The studies of Moore (1989) in the identification of the exchange between 
members, models of communication by Shannon and Weaver (1949) and interaction as a social and psychological 
connection by Zhang and Fulford (1994) are just some of the research that has been conducted concerning 
interaction and communication (Roblyer & Wienke, 2003). 

Conclusion 

This study provided a preliminary review of the literature concerning quality in distance education. This review 
provides some of the major themes and thoughts concerning the area of quality in distance education. Currently the 
areas of evaluation are at the institutional, program, course, instructor and student levels. Evaluation methods at the 
institutional level are more aligned with state and federal compliance for accreditation. Course and program 
evaluation are geared toward improving the design and/or delivery of on-line instruction. Faculty and student 
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evaluations model the traditional classroom evaluations, with some attempts to evaluate learning styles and 
interaction during on-line instruction. 

The definition of quality varies and is often due to the area being evaluated and the nature of the study. 
Recommendations for the standardization of quality and the criteria used to determine quality in on-line instruction 
is noted along with the partnering of business and government to ensure quality in training America’s future 
workforce. 

Implications for Research and Practice 

Although the results of this preliminary literature review provided articles concerned with quality in higher 
education, current indicators state that “80 percent of Fortune 500 companies are using or intending to use, e- 
learning, and expect a significant ROI” (Macpherson, Elliot, Harris, & Homan, 2004, p. 297). Since trends indicate 
that distance education is on the rise and will continue to do so, further research on the development of applicable 
and effective evaluation instruments for measuring quality in on-line learning is essential. Evaluation has been 
studied by various entities, professionals and researchers (Wentling & Johnson, 1999). It is this foundational 
knowledge that can aid in producing new methods for pre-evaluation of quality centered on the student/employee. 
These instruments can then aid practitioners and students in obtaining quality on-line training, courses or programs 
that are beneficial in promoting learning and performance. This is particularly important since the American Society 
for Training and Development indicates that e-leaming (distance education) usage is increasing with learning 
technologies delivering close to 10 percent of training programs (Gallaher, 2003). 

Another area for research is determining a common definition and/or criteria for quality. The standardization of 
quality in distance education would assist in decreasing atheoretical practice. According to Swanson and Holton, 
(2001) “the improvement ideas of making positive change, attaining expertise, developing excellent quality, and 
making things better are central to HRD. This core goal of improvement is possibly the most important idea in the 
profession and the core motivator of HRD professionals” (p. 15). If this is truly a core goal, then it is essential for 
HRD to formulate a definition for quality and supply a tool to determine quality programs and training to guide 
practice. 

Contributions to New Knowledge in HRD 

Traditional evaluation methods have always been after a course and/or training. The concept of developing a pre- 
evaluation instrument for determining quality in a distance education training, course or program provides HRD 
practitioners and researchers a tool to continue to increase the learning and performance at the organizational, group 
and individual level. This joint effort could aid in bringing HRD academicians and scholarly practitioners into the 
forefront of knowledge creation relevant to the theory and practice for the field of HRD. “Without a systematic and 
comprehensive evaluation, it is hard to see how e-learning (distance education) as an HRD strategy can be 
developed to ensure the delivery of quality human resources so important to organizational strategy” (Macpherson, 
et al, 2004, p. 307). 

Additionally HRD can assist in defining the criteria for quality in distance education, thus promoting it’s stance 
on value and ethics for the benefit of the organization, group, and individual. HRD will need to retain its’ core value 
of connecting with human beings and in connecting human beings in a meaningful way through the use of quality 
distance education for educational human resource development (Swanson & Holton, 2001). 
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